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1.0 Introduction 

In-flight icing is a significant hazard for the aviation indus- 
try. It occurs when supercooled liquid water (SLW) comes in 
contact with, and freezes to, the leading surfaces of an aircraft. 
This can significantly alter the aircraft’s aerodynamic 
properties by increasing the amount of drag on the aircraft, 
and reducing the lift. Since practical airborne remote detection 
hardware has not yet been developed, a ground-based 
detection system that can provide information to all aircraft 
entering and departing a terminal area (Fig. 1) is a key element 
in facilitating icing avoidance (Serke et al., 2010). 

Currently there are two systems that are being developed for 
the detection of in-flight icing. The first detection system is the 
NASA Icing Remote Sensing System (NIRSS) a testbed that 
integrates three vertically pointing sensors; a Vaisala Laser 
Ceilometer, a Metek K a -band radar, and a Radiometrics Corpora- 
tion 23-channel radiometer (Fig. 2), (Reehorst et al., 2006). 

The multichannel microwave radiometer has the ability to 
derive integrated liquid water (ILW), atmospheric water vapor 
and temperature profiles (Solheim et al., 1998). A Vaisala laser 
ceilometer is used to define cloud base heights, and a Metek Ka- 
band radar is used to delineate cloud top and base heights. 
NIRSS combines the ILW, radar reflectivity, temperature 
profile, and cloud top and base heights to determine the 
presence of in-flight icing conditions in the atmosphere. 

The second in-flight icing detection system is the Current 
Icing Product (CIP) which was developed at the National 
Center for Atmospheric Research. This system combines 
visible and infrared satellite imagery, radar reflectivity, 
lightning observations. Pilot Reports (PIREPs) and standard 
ground-based weather observations with numerical model 
output to produce a gridded, hourly, three dimensional 
representation of icing probability and severity (Bernstein et 
al., 2005). Each horizontal grid point of CIP is based on a 
20 km by 20 km Rapid Update Cycle (RUC) model grid point. 
First developed during the winter of 1997/98, CIP became an 
operational National Weather Service product in 2002. 

Icing-related PIREPs are voluntary reports made by pilots to 
report on the presence or absence of in-flight icing conditions 


and other weather-related conditions. Both a subjective icing 
severity and icing type (rime, clear or mixed) are included. 
PIREP reports of no icing are useful as well, since the absence 
of icing is important information. The shortcomings of PIREPs 
are well documented and include non-uniformity in time or 
space and contamination by errors in location, altitude and time 
(Brown et al., 1997; Kelsch and Wharton, 1996). PIREPs can 
sometimes be inaccurate due to time lags before the pilot reports 
the observed icing condition, and whether he or she reports the 
correct altitude and location. The reported severity is also 
somewhat subjective as it can vary based on aircraft type, phase 
of flight, and pilot experience. Nevertheless, PIREPs are our 
only means of in-situ diagnoses of actual atmospheric condi- 
tions encountered by pilots and their aircraft in the absence of 
expensive icing research flights or specially instrumented fleet 
aircraft. The objective of this study is to examine how the 
testbed NIRSS icing severity product and the operational CIP 
severity product compare to PIREPs of icing severity, and how 
the NIRSS and CIP compare to each other. 

2.0 Methodology 

A 3-year database of CIP, NIRSS and PIREP data was 
compiled focusing on winter periods from early November 
2008 to late March 2010. During these three winter seasons, 
917 icing PIREPS were collected within 40 km of the NIRSS 
system located in Cleveland, Ohio. CIP icing severity output 
from the nearest RUC gridpoint to the NIRSS location was 
archived and icing severity values were extracted at the time 
and height of each icing PIREP. A similar process was 
conducted for NIRSS severity output. If there was no icing 
data at the exact PIREP altitude, a vertical search was 
performed for the nearest altitude with icing severity data. 
Once this temporal and spatial matching was completed for all 
PIREPs, a statistical comparison was begun. Analysis 
occurred from the ground level to ~ 30,000 ft (or 9,144 m). 
For this study, a PIREP reported over a range of heights is 
treated as multiple PIREPs spread over 1000 ft increments 
(Wolff et al., 2010). 
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3.0 Analysis and Discussion 

3.1 Case Study Comparison — December 15, 
2009 

An example icing case study is presented here to illustrate 
how the comparison of NIRSS, C1P and PIREP severity 
looked for a single event. In the next section, statistics for 
3 years of such cases are discussed. 

On December 15, 2009, at 0000 UTC, a surface low was 
dominant over the central Great Lakes region (Fig. 3). The 
warm front extended from the southeast portion of Lake 
Huron eastward into southeastern New York. The cold front 
was oriented from north central Ohio through southwest Ohio. 
The cloud top temperatures over the Cleveland area were 
between -5 and —15 °C (color scale). This temperature range 
has been shown in previous research to be conducive to 
supercooled liquid water (Rogers and Floyd, 1989). 

At approximately 0300 UTC, the cold front passed through 
Cleveland, Ohio. In the hours following the cold frontal 
passage, drizzle and rain fell over the metropolitan area, which 
changed to snow by 1500 UTC. 

Figure 4 shows time versus height of icing severity from 
N1RSS (top), PIREPs (top numerals) and CIP (bottom) for 
December 15, 2009. A zero to eight scale for icing severity 
was used for these plots, and the comparisons throughout the 
rest of this study where zero is no icing, one is trace amount of 
icing, two to three is light, four to five is moderate and six to 
eight is heavy. 

Twenty-eight positive icing PIREPs were recorded between 
1100 and 2200 UTC around Cleveland. In addition, three 
negative PIREPS were recorded during this time period. 
N1RSS diagnosed significant icing between 0900 and 
2400 UTC from 1 to 6 kft AGL. CIP diagnosed icing from 
0900 to 2400 UTC as well, from roughly 1 to 9 kft AGL. For 
this case, the temporal variability and the magnitude of the 
severities generally match between the two products despite 
the fact that CIP has a 1-hr time resolution and NIRSS has 
one-minute resolution. CIP has a conservative cloud top and 
base estimate scheme (done purposely to insure thorough 
warnings). 

3.2 3-Year Archive Comparison 

In the previous section we explored the comparison of icing 
products for a day-long icing case. This section will be a 
statistical intercomparison of all three icing detection methods 
for the full 3 -year study period. Similar to the case study 
presented above, the closest NIRSS and CIP icing severity 
measurements were found to each of the 917 PIREPs within 
40 km of the NIRSS location. The matched severity categories 
were plotted in Figures 5 to 7. 


Figure 2. — Image of the NIRSS hardware located at the NASA 
Glenn Research Center in Cleveland, Ohio. 


Figure 3. — 0000 UTC surface pressure (yellow lines, [mb]), 
cloud top temperature (color bar, [°C]) and frontal analy- 
sis for December 15, 2009. 
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Figure 4. — Time [hh] versus height [kft] plots of NIRSS (top, color scale), PIREP (top, red numerals) and CIP (bottom, 
color scale) icing severity from December 1 5, 2009 from 0000 UTC to 2400 UTC. 


NIRSS versus PIREP severity is shown in Figure 5, with 
one-to-one severity correlation bins highlighted in orange. The 
numbers in each bin represent the total number of icing 
severity matchups recorded for the 3 -year period. A linear 
best-fit line is overlain in blue. Taking the square root of the 
resulting R-squared value gives NIRSS a severity category 
correlation coefficient of 0.35 to the PIREP severity category. 
NIRSS appears to do well locating negative PIREPs, as well 
as finding moderate icing PIREPs. Very few severe or heavy 
PIREPs were reported during this time period. There are a 
significant number of positive PIREPs that NIRSS identifies 
as negative severity, possibly due to the high time resolution 
of NIRSS’s ILW algorithm when viewing localized SLW 
cases. 

CIP versus PIREP severity is shown in Figure 6. A linear 
best-fit line is again overlain in blue, with CIP having a 
severity category correlation of 0.21 to the PIREP severity 
category. CIP and NIRSS both do well at finding PIREPs from 
light to moderate values. CIP seems to correctly identify a 
much smaller fraction of negative PIREPs than NIRSS. This 
could be a result of the conservative cloud base and top 
diagnosis in CIP. 

NIRSS versus CIP severity is shown in Figure 7. The opera- 
tional CIP product is treated as ‘truth’ in this comparison, as 


NIRSS is still a testbed. A linear best-fit line is again shown in 
blue, with NIRSS having a severity category correlation of 
0.18 to the CIP severity category. There seems to be a 
significant spread in the collocated severity values between 
the two products during the 3 -year study period. This spread is 
likely due to the difference in temporal resolution of the 
products, and the fact that the two products arrive at hazard 
estimates based on different input datasets. 
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Figure 5. — Overall PIREP severity versus NIRSS severity. 
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Figure 6. — Overall PIREP severity versus CIP severity. 
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Figure 7. — Overall CIP versus NIRSS severity. 


Another useful statistic is the probability that each product 
will detect negative and positive icing PIREPs, or the 
Probability of Detection (POD). These statistics are termed 
POD n and POD y , respectively. To get POD n , the fraction of 
negative icing PIREPs that the respective product identifies as 
negative icing is determined and then divided by the total 
cases where PIREP severity is equal to zero (Eq. (1)). 


POD n — (Total Cases product icing severity = 0 when PIREP severity = 0) (1) 

(Total cases PIREP icing severity = 0) 


Similarly, POD y is the fraction of positive icing PIREPs of any 
category (one through eight) that the respective product 
identifies as positive icing divided by the total cases where 
PIREP severity is greater than zero (Eq. (2)). 


PODy — ( Total cases product icing severity > Q when PIREP severity >0) (2) 

(Total cases PIREP icing severity >0) 


For the 3-year study period, POD y and POD,, were calcu- 
lated for the product comparisons shown in Figures 5 to 7. The 
results are shown in Table 1. 


TABLE 1.— OVERALL POD„ AND POD y STATISTICS 
[N versus P (NIRSS versus PIREPS), C versus P 
(CIP versus PIREPS), N versus C (NIRSS versus CIP)] 



N versus P 

C versus P 

N versus C 

POD v 

0.78 

0.90 

0.79 

POD,, 

0.71 

0.29 

0.59 


NIRSS detected greater than 70 percent of both positive and 
negative PIREPs. CIP detected 90 percent of positive PIREPs 
but only 29 percent negative PIREPs. The percentage of the 
time averaged vertical profile that a product has identified a 
positive severity value is termed the warning volume. The 
warning volume for this study is calculated from the surface to 
the average height of the tropopause. A successful product 
must find an optimal balance between POD yes and no and 
warning volume because it would not be very useful for a 
product to have a POD y of 1.0 (perfect icing detection) if the 
entire column is warned on at all times. Ideally, a product 
would have a maximized POD y and POD„ with a minimized 
warning volume. For the 3-year study period, NIRSS had a 
mean warning volume of 13 percent, and CIP had a mean 
warning volume of 34 percent. CIP detected 10 percent more 
positive PIREPs in over twice the warning volume. Further- 
more, CIP’s high warning volume causes it to classify regions 
as positive icing where they should be devoid of icing, based 
on negative PIREPs. 


Summary 

In-flight icing detection is crucial to achieving a high safety 
standard for the national fleet of commercial and general 
aviation aircraft. In this study a comparison was done showing 
the quantitative severity categories of negative and positive 
icing PIREPs to the quantitative icing severity derived from 
the prototype NIRSS icing detection algorithm and operational 
CIP icing algorithm. An icing case study from December 15, 
2009, over the NIRSS location in Cleveland, Ohio, was 
discussed to illustrate how the PIREP and icing product 
severities were compared. A statistical analysis over the full 
3-year study period found that NIRSS detected in-flight icing 
and negative icing at least as good as CIP when compared to 
all PIREPs within a 40 km radius. NIRSS detected almost 
80 percent of positive PIREPs and over 70 percent of negative 
PIREPs in a relatively smaller warning volume. CIP detected 
slightly more positive PIREPs than NIRSS but did fairly poor 
in detecting negative PIREPs. This occurred in a warning 
volume over twice the percent of NIRSS’s warning volume. 
CIP did very well at detecting positive PIREPs. NIRSS 
displayed respectable probabilities of icing detection with 
lower warning volumes than CIP. This is due to NIRSS having 
a higher time resolution and utilizing physically based vertical 
profiles of ILW, temperature and radar reflectivity. Therefore, 
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the NIRSS testbed in-flight icing severity product seems to be 
at least as good as CIP. A shortfall of NIRSS is that it 
currently lacks volumetric scanning capability. This is being 
addressed by the addition of a 1° beamwidth multichannel 
scanning radiometer (Serke et al., 2010). Future work with 
NIRSS will include exploring Doppler fall velocities to detect 
possible freezing drizzle and freezing rain, and comparing 
NIRSS hazard detection to the polarimetric data from future 
upgraded NEXRAD. 
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